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Problem / Background / Context: 

Description of the problem addressed, prior research, and its intellectual context. 

Over the past 20 years, gaps in students’ educational aspirations have shrunk considerably 
(Roderick et al., 2008; Roderick, Nagaoka, & Coca, 2011; Kao & Tienda, 1998; Avery & Kane, 

2004; Turner, 2007; Deil-Amen & Tevis, 2010). Similarly, racial and ethnic gaps in college 
enrollment have shrunk. The one area in which postsecondary racial gaps have not improved is 
in college completion (Turner, 2007). Given the complex and varied nature of the college 
application, enrollment, and graduation process, it is unsurprising that district and school policies 
have used a scattershot approach to evaluating college readiness, with rubrics containing items 
pertaining to non-cognitive skills, test scores, numbers of applications, FAFSA completion, 
college visits, and college knowledge. The problem with such an approach is not that these items 
are bad in and of themselves - indeed, all of these factors have valid research behind them - but 
rather there is no sense of the relative importance of these factors nor of the degree to which any 
of these factors ultimately affects degree completion. 

Purpose / Objective / Research Question / Focus of Research: 

Description of the focus of the research. 

While this project was originally started to answer the question “What are the factors that make 
up college readiness?” our findings quickly pointed to the fact that this question is meaningless 
outside of the context of the question “ready for what?” That is, effects of postsecondary 
institutions on students’ likelihood of degree attainment is so large that the idea of college 
readiness being assessed in high school (without the knowledge of what institution a student will 
attend), is essentially meaningless. As a result, the purpose of the project shifted away from a 
strict focus on high school determinants of degree completion, and towards an inquiry focused 
on measuring the effects that postsecondary institutions have on students’ likelihood of degree 
completion. At the same time, our primary purpose as a research organization is to serve Chicago 
Public Schools, meaning that the main audience for this work is practitioners and administrators 
in secondary schools. Therefore, we have framed the analysis in terms of which factors, as 
viewed from a high school standpoint, can have the largest influence on degree attainment. We 
propose four possible policy options: Raise Grades (GPA), Raise test scores (ACT), improve the 
rates at which students enroll in ‘match’ colleges, improve college choice by looking at colleges’ 
institutional four-year graduation rate. 


Setting: 

Description of the research location and partners involved, if applicable. 

This study focuses exclusively on Chicago Public Schools (CPS) seniors. CPS is the third largest 
school system in the country, with 122 high schools, and approximately 400,000 students per 
year. Currently, approximately 45% of graduating seniors enroll in a four year college the fall 
after graduating. Ten colleges account for 50 percent of college enrollment for CPS graduates, 
and nine of these colleges are located in the Chicago metropolitan area. The five most popular 
colleges attended by CPS graduated are all Illinois public colleges. With the exception of the 
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state’s flagship university, University of Illinois at Urbana-Champaign, all Illinois public 
colleges are considered Somewhat Selective in our rubric of college access, based on Barron’s 
rating guide. 

While a relatively small number of colleges can account for half of all CPS college enrollments, 
in a typical year, CPS students attend another 470 colleges and universities around the country. 
These colleges range from Very Selective to Non Selective and have graduation rates ranging 
from 20% to 95%. 


Population / Participants / Subjects: 

Description of the participants in the research: who, how many, key features, or characteristics. 

For this study, we focus on the graduating high school cohorts of 2003 - 2005. Going this far 
back in time allows us to examine these cohorts’ college four and six year graduation rates. Our 
analytic sample is comprised of 42,129 graduating seniors, 14,032 of whom enrolled in a four 
year college the fall after high school graduation. Among students who enrolled in college, the 
average ACT score was 20, and the average GPA was 2.77 and these students graduated at a rate 
of 23.8%. 

Research Design: 

Description of the research design. 

This study focuses on identifying key predictors of college completion. Once these key levers are 
identified, we run large scale simulations to get a picture of which policy interventions, if 
successful, might have the biggest effect on increasing college graduation rates. For example, 
given the enormous focus currently placed on standardized test scores, what would be the effect 
of increasing all ACT scores by a single point on system wide college graduation rates? 
Alternatively, are there more returns to increasing student grades? Furthermore, recent research 
published by Roderick et. al. (2007), as well as Bowen and McPherson (2009), points to the 
importance of college match for college success. If all students went to a match college, how 
much would graduation rates increase? Or, is it possible that in Chicago, students are simply 
going to colleges that are worse than average in terms of institutional graduation rate? By 
measuring which levers can influence the biggest change, we can frame the policy debate around 
getting more students to attain their goal of completing a college degree. While certainly not a 
causal analysis, these simulations represent the largest possible impact that such interventions 
can have and hence provide a realistic set of expectations that administrators can use to set 
reasonable but ambitious goals. This simulations approach is recommended by Zellner (2009), 
and similar approaches have been used to examine potential effects on college outcomes of changing 
Affirmative Action policies (Espenshade & Chung, 2005; Archidiacono, 2005), college costs and 
family environments (Cameron & Heckman, 2001), and financial aid packages (Desjardins & 

McCall, 2010). 


Data Collection and Analysis: 

Description of the methods for collecting and analyzing data or use of existing databases. 

This study uses administrative data from the Chicago Public Schools, archived by the 
Consortium on Chicago School Research at the University of Chicago. The current study uses 
data on students’ backgrounds, neighborhood census data, test scores, and course grades, as well 
as school compositional data. As a part of the state accountability system, all students take the 
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ACT in the 1 1th grade. In order to identify whether graduates enroll in college in the fall after 
graduation, the kinds of colleges they attend, and their graduation rates, this study uses data from 
the National Student Clearinghouse (NSC). In 2004, NSC expanded its services to high school 
districts through its new program, Success Outcomes. CPS is the first major urban school district 
to participate in this program and produce reports on its graduates. Currently, NSC’s enrollment 
verification program covers 92 percent of postsecondary enrollment in the United States. 

Our analysis strategy for this project relies on building large scale Hierarchical 
Generalized Linear Models (HGLM) as well as a parallel specification using fixed effects. The 
HGLM allows us to model individual student characteristics at level 1 while also controlling for 
school effects at level 2. By controlling for factors at both levels we are able to develop a decent 
model for explaining variation in degree completion across students. Most importantly we are 
able to adjudicate between how well test scores and grades predict college graduation while 
simultaneously retaining our ability to examine high school level effects such as whether an A 
from one school means the same thing as an A from another. We then complicate this model by 
adding in college fixed effects in an attempt to measure the effect of colleges on students’ 
likelihood of graduation. This second model informs two of our policy options. To model the 
effect of moving all students to a match college, we identify all students who have attended an 
‘undermatch college,’ that is, students whose qualifications give them access to a more selective 
college than they attended. We then assign these students the fixed effect of a random match 
college 1 . By looking at the difference in predicted probability of graduation for the sample after 
college reassignment, we are able to get a best case scenario prediction of the effect of getting all 
students to match. 

To run our second college simulation, we calculate predicted probabilities of graduation 
controlling for but not including the college fixed effects. Essentially, this gives us the predicted 
probability that a student has of graduating from college at the end of high school, sans any later 
positive or negative effect their college has on that probability. Then, rather than grouping 
colleges by selectivity we group students according to their predicted probability (dividing those 
probabilities into 0.10 bands, e.g. 0.101 to 0.200). We then examine the college graduation rates 
for the students in each probability band the colleges that they attend, allowing us to compare the 
outcomes of similar students at different colleges. Given the variation we see in in these college 
graduation rates, we examine the effect of improving the graduation rate of all students who 
attend colleges whose graduation rates are below average for their group and see what the effect 
is across all groups. 

Findings / Outcomes: 

Description of the main findings or outcomes, with specific details. 

In looking at the effects of student level characteristics that determine college graduation, 
academic factors are extremely important (See Table 1 for model output). Specifically, GPA and 
ACT are both equally strong predictors of enrolment in college. Looking further however, we 
find that for college graduation GPA’s effect is much larger than ACT’s and while ACT’s effect 
has marginal decreases (the higher one’s ACT, the less each additional point influences 
graduation) GPA has in increasing marginal effect. That being said, most CPS students graduate 
with such low qualifications that moderate improvements in qualifications are not yield relatively 


1 We only allow students to be assigned to a college where at least 20 students from our sample already attended. 
This allows us to have confidence that the fixed effect is reasonably robust. 
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small, but significant, changes in their likelihood of graduation. Among all graduates, the 
average GPA is 2.3 and the average ACT score is 18. Even among college-goers, these 
qualifications are still quite low, with averages, respectively, of 2.75 and 20. These are both 
lower than national averages. 

The results from our college level simulations (Table 2) are even more surprising. Overall 
improving moving all students to a match college improves the degree attainment rate for CPS 
college-goers by 4 percentage points, approximately halving the gap between CPS and the 
nation. While this is impressive, these effects are not equally distributed across the population. 
Students with high qualifications upon exiting high school benefit disproportionally from this 
intervention. Part of this effect is tautological. Students who only have access to non-selective 
colleges are matching so long as they are in college, so no movement under this intervention is 
possible. However, the results of this analysis clearly point to the significant cost of high 
achieving students attending colleges below the level of selectivity to which they have access. 

Our final simulation yields similar overall results with a different distribution of effects. 
The effect on the system- wide degree attainment effect would be the same using this 
intervention, but the effects are concentrated in the middle of the achievement spectrum. Given 
that this is where the majority of our students lie, there are clear implications for the work of 
teachers and counselors who are advising students where to attend school. Perhaps most clear in 
this analysis is the existence of schools which seem to actually harm students’ likelihood of 
completing a degree. Furthermore, our analysis points to the fact that schools and districts do not 
need create complicated models of predicted probability of graduation in order to advise 
students. Across the achievement spectrum, the publicly reported four- year graduation rate of 
colleges is strongly correlated to CPS students’ observed graduation rates. This implies that 
students should always try and choose the college with the highest reported four- year graduation 
rate to maximize their own likelihood of graduation. 

Conclusions: 

Description of conclusions, recommendations, and limitations, based on findings. 

The conclusions for these findings are stark. While high schools, would be wise to focus 
on increasing GPA’s, these improvements in student achievement are for naught if students make 
college choices that lead them to low performing colleges and universities. Similarly, it is clear 
that while some colleges are able to ensure that nearly all students who are accepted graduate, 
there are other schools where only 20% are able to graduate within six years, leading one to 
question: what must a student do to be adequately prepared for an institution where 80% do not 
succeed? Our findings strongly suggest that the wide variation in college graduation rates, even 
among similarly qualified students, make it impossible to set a single set college readiness 
benchmarks. It does appear possible, however, to create a demarcation of responsibility between 
high schools and colleges. On the one hand, high schools could be doing a great deal more to 
boost student achievement, especially related to grades, and providing structured supports around 
these students to ensure that they apply, and enroll in colleges that maximize students’ chances 
of success. Colleges, on the other hand, play their own role in supporting students through 
graduation. 

These findings challenge both high schools and colleges. For high schools, the importance of 
making hard work pay off lies in helping students make college choices that translate their high 
school preparation into the highest chance of earning a college degree. For colleges, the 
challenge in making hard work pay off is to provide the supports necessary for the high- 
achieving students they admit to continue to succeed on their campuses. 
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Appendix B. Tables and Figures 
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Table 1) Logistic regression of the effects of academic qualifications on four-year college attendees’ 
probability of graduating in four years, including fixed effects of students’ college. 



Coefficient 

Standard Error 

High School GPA 

-.017 

.315 

GPA 2 

.016 

.054 

ACT score 

.014** 

.047 

ACT 2 

-.005*** 

.001 

GPA* ACT 


.011 

Female 

.285*** 

.046 

White 

423 *** 

.073 

Latino 

.132 

.070 

Asian 


.079 

Concentrated Poverty 

-.058*** 

.032 

Social Status 

.112 

.315 


Table 2) Predicted increase in graduation rate after simulating increases in students’ college choices. 



Original Graduation 
Rate 

No Students 
Undermatch 1 

No “below average” 
colleges 1 

All Students 

26% 

4.0 

5.1 

Very Selective Access 
8% 

63% 

8.6 

4.1 

Selective Access 
24% 

33% 

6.4 

5.9 

Somewhat Selective 

Access 

41% 

14% 

2.0 

5.2 

Nonselective Access 
24% 

7% 

O 2 

3.8 

Note. 1 Column represents the percentage points ad 

Ided to the predicted graduation rate under each 


simulation. 2 By definition, all students with nonselective access are at least matching in their college 
choice by attending any four-year college, regardless of selectivity level. 
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